Morphologically Aware Word-Level Translation

We propose a novel morphologically aware probability model for bilingual lexicon induction, which jointly models lexeme translation and inflectional morphology in a structured way. Our model exploits the basic linguistic intuition that the lexeme is the key lexical unit of meaning, while inflectional morphology provides additional syntactic information. This approach leads to substantial performance improvements - 19% average improvement in accuracy across 6 language pairs over the state of the art in the supervised setting and 16% in the weakly supervised setting. As another contribution, we highlight issues associated with modern BLI that stem from ignoring inflectional morphology, and propose three suggestions for improving the task.Comment: COLING 202

UniMorph 4.0:Universal Morphology

The Universal Morphology (UniMorph) project is a collaborative effort providing broad-coverage instantiated normalized morphological inflection tables for hundreds of diverse world languages. The project comprises two major thrusts: a language-independent feature schema for rich morphological annotation and a type-level resource of annotated data in diverse languages realizing that schema. This paper presents the expansions and improvements made on several fronts over the last couple of years (since McCarthy et al. (2020)). Collaborative efforts by numerous linguists have added 67 new languages, including 30 endangered languages. We have implemented several improvements to the extraction pipeline to tackle some issues, e.g. missing gender and macron information. We have also amended the schema to use a hierarchical structure that is needed for morphological phenomena like multiple-argument agreement and case stacking, while adding some missing morphological features to make the schema more inclusive. In light of the last UniMorph release, we also augmented the database with morpheme segmentation for 16 languages. Lastly, this new release makes a push towards inclusion of derivational morphology in UniMorph by enriching the data and annotation schema with instances representing derivational processes from MorphyNet

Morphological competence in neural natural language processing

In case-marking languages (CMLs), such as Polish or Finnish, a substantial portion of grammatical information is expressed at the word-level. The word-forms provide information about their inherent properties, like tense or mood, but also encode information about relations between the words. This is in contrast to morphologically impoverished languages, like English, which undergo little inflection. The linguistic factors associated with CMLs make them a challenge for data-driven, neural natural language processing (NLP). To successfully process a CML, neural NLP models must be morphologically competent; i.e., they have to capture both the meaning and function of different components of a word form and recognise the importance of morphological signals within a language. Despite the importance of morphological competence for language processing, the neural NLP models have never been directly tested for that linguistic ability. This gap in the literature is the more important given that most neural NLP models are developed with English language in mind and later applied, without any adaptations, to other languages. It remains unclear whether the architectures and optimization techniques developed on English are able to extract all the essential information from the word-forms of CMLs and whether they can interpret this information at the clausal-level to solve NLP tasks. In this thesis I investigate whether state-of-the-art neural models for CMLs utilise morphosyntactic information when solving a task for which this information is key: dependency parsing. To answer this question I propose a new evaluation paradigm which involves evaluating the models on various counterfactual versions of dependency corpora. Through evaluation of Polish, Russian, Finnish and Estonian dependency parsers, I reveal that the models often fail to recognise morphology as the primary indicator of syntax; instead of generalising based on the case and agreement markings, they learn to over-rely on word order and lexical semantics. Following this finding, I experiment with two methods of increasing the models' reliance on morphology: one based on the alteration of the training data and another involving an enhanced training objective. Finally, through creating synthetic CMLs by manipulating selected typological properties of Polish, I investigate whether the models have a 'preference' for the means of encoding case information and reveal that syncretism and high fusion are amongst the properties that drive the models away from relying on morphology as a signal to subject/objecthood.Vice-Chancellor’s & Selwyn College Scholarship and funding from University of Cambridge the Department of Technology and Computer Scienc

Morphologically Aware Word-Level Translation

We propose a novel morphologically aware probability model for bilingual lexicon induction, which jointly models lexeme translation and inflectional morphology in a structured way. Our model exploits the basic linguistic intuition that the lexeme is the key lexical unit of meaning, while inflectional morphology provides additional syntactic information. This approach leads to substantial performance improvements—19% average improvement in accuracy across 6 language pairs over the state of the art in the supervised setting and 16% in the weakly supervised setting. As another contribution, we highlight issues associated with modern BLI that stem from ignoring inflectional morphology, and propose three suggestions for improving the task